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The concepts and methods are reviewed which have proven to be of the most value in designing and 
implementing the Octopus computer network, which is one of the largest concentrations of computing 


capability in the world. 


The discussion summarizes design principles relating to the scope of system 


software, privacy and security, processor organization, file structure, network connections, hard- 


ware selection, programming techniques, standards, and use of resources. 


Differences with what 


appear to be widespread belief and practice are cited, including such matters as the size of the 
programmer staff, the significance of the privacy issue, the importance of the choice of languages 
and language constructions, the primary causes of system failure, and efficiency. 


1. INTRODUCTION 


The Octopus computer network, designed and imple- 
mented at the Lawrence Livermore Laboratory of the 
University of California (LLL), constitutes one 

of the largest (if not the largest) concentrations 
of computing power and storage capacity in the 
worldI1], The network currently interconnects four 
C.D.C. 7600 computers; two C.D.C. STAR-100 computers 
are being added at the present time. On-line 
storage includes the 10**~bit 1.B.M. Photodigital 
store, while off-line storage is concentrated on 
over 30,000 magnetic tapes. Input-output capabil- 
ity includes two 18,000 line/minute printers, two 
I.I.I. FR-80 microfilm recorders, Evans and Suther~ 
land LDS-1 displays, over 200 television monitors, 
and over 750 teletypewriters. The network has 

been in a continual state of change and growth for 
nearly a decade, new hardware being added as 
technology advances; as a consequence the system 
usually has included the fastest and largest 
capacity components currently available. 


Octopus supports over 1500 interactive and batch 
users, and it is not uncommon to find 250 of them 
logged into the system at one time. The programs 
executed for these users range from highly inter- 
active text editors and file query routines to 
huge numerical simulations which can execute for 
over ten hours on a C.D.C. 7600 and generate output 
filling 20 or more reels of magnetic tape. A 
typical daytime hour will see tens of thousands of 
messages pass through the network, including over 
500 log in messages. 


The preceding brief summary should make it clear 
that Octopus constitutes a very significant com- 
puter system implementation. It would seem there-— 
fore that any observations made or conclusions 
drawn about network and operating system design 

and implementation, which derive from experience 
with Octopus, should be of general interest. This 
is particularly so since those observations and 
conclusions are not always in agreement with widely 
accepted views and practice. The remainder of this 
presentation will summarize several principles 
based on Octopus experience. Since it is of course 
not true that all persons who work on Octopus share 
identical views, the statements made here under 
editorial we at best represent a consensus and of 
necessity reflect personal opinions of the. present 
author. 


*Work performed under the auspices of the Energy 
Research and Development Administration. 
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2. SYSTEM FUNCTIONS 


Probably the most fundamental principle of all is 
that the system be kept as small as possible. 
System is here defined as the set of routines which 
have full access to the capabilities of the pro- 
cessor hardware, i.e., what is often called the 
kernel of the system. In the case of computers 
that execute user programs (which are called workers 
in Octopus terminology and include the large C.D.C. 
computers listed earlier), system routines execute 
in a privileged mode not available to users' pro- 
grams. The system should perform those functions, 
and only those functions, which if a user were 
allowed to perform them would permit him to damage 
the system, to disturb other users, or to gain 
special privilege for himself. The system performs 
file accesses, controls the flow of messages through 
the network, allocates resources, carries out all 
I/O activity, interprets requests made by user 
programs, and handles all matters relating to 
accounting and security. The system does not 
include utility routines (compilers, interpreters, 
relocatable loaders, text editors, information 
retrievers, debuggers, etc.) nor of course does it 
include any application programs, 


The importance of thus limiting the system is that 
it keeps the network design manageable. Because 
of the size of Octopus the system cannot be small, 
but if it included unnecessary functions it would 
become too large to keep well organized. Having a 
system with limited functions also allows the 
programming load to be distributed more widely. 
example, the responsibility for programming a 
specialized compiler or other routine of interest 
only to a small group of users can reasonably be 
placed on that group and not on a central staff. 


For 


The effect of this principle is that the system 
programming staff at LLL has over the years averaged 
about 30 persons. This figure includes those who 
program the most widely used compilers (e.g., 
Fortran, Algol, Cobol) and other utility routines 
as well as those who program the system proper. 
Further, this staff is responsible not only for 
maintaining and modifying the system, it has 
designed the entire network from scratch. (Octopus 
does not employ manufacturers’ or other commercial 
software, a primary reason being that that software 
neither adheres to the principles outlined here not 
provides for network activity, and is therefore 
inadequate for LLL needs[2])}, It is so clear from 
the Octopus experience that one or two persons are 


sufficient to program a minicomputer and that no 
more than half a dozen are needed for a worker 
computer operating system or a major compiler that 
we find it difficult to imagine how tens or hundreds 
of persons are kept busy on similar projects at 
other installations. 


3. PRIVACY 


It is really the issue of privacy that dictates 
which functions are assigned to the system. It 

is undesirable that the carelessness (or mali- 
ciousness) of one user interfere with the activity 
of another user or allow unauthorized access to 
that user's private data base. All system routines 
must be written with the intent of assuring privacy; 
we have found that this is not difficult to do. 
Except for the questions of programmer oversights 
and careless implementation, Octopus software 
appears to be entirely secure. A good analogy that 
has been suggested in regard to this is that it is 
not difficult to build a leak-proof container for 
liquid if one uses solid plates and not wire mesh 
in the design. The analogy suggests, moveover, 
that it may not be easy to fix the leaks in an 
improperly designed and already implemented system. 
The surprising thing is that so many commercially 
available systems are built of wire mesh. 


Briefly, Octopus uses a combination (or password) 
to verify that a user logging in is in fact who 
he claims to be. Thereafter, the system relies 
on records associated with that user to determine 
which files and other resources may be made avail~ 
able to hin. 


4. SYSTEM ORGANIZATION 


The network is fully interactive, the worker com- 
puters being time~shared and multiprogrammed. The 
vastly reduced turnaround time obtained by such 
operation very significantly improves efficiency 
from the users' viewpoint. Batch~like operation 
is included as a special case. 


Processor hardware permitting memory partitioning 
and automatic program relocation is essential. 
Only with such hardware can the users be permitted 
to program freely in whatever language they choose 
and still execute their programs at full processor 
speed. Denying such capability would constitute 
an intolerable burden on the users. More advanced 
memory hardware (such as two-segment relocation or 
paging) which permits reentrant programs and other 
forms of memory sharing has been found to be a 
worthwhile improvement over simpler schemes, since 
more efficient utilization of memory is possible. 


The central Octopus filing system, called Elephant, 
which uses the 10!?~pit Photostorel3J, is accessed 
through a system of directories . A directory is 
a special kind of file which associates pointers to 
other files with symbolic names for those files; 
the files so listed may be either simple (data or 
program) files or other directories. The entire 
directory structure is of the form of a general 
directed graph. A user can access only that por- 
tion of the structure accessible by a chain of 
pointers starting at a root directory associated 
with him. This scheme has proven a considerable 
success, particularly since it provides the 
opportunity for very general information sharing 
arrangements among users, as well as permitting 
each user to logically structure his data in a 
convenient way. 


5. NETWORK ORGANIZATION 


Octopus is currently organized as a superposition 
of partially independent subnetworks that use store 
and forward protocol. Each subnetwork is centered 
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on a minicomputer, the concentrator of the subnet- 
work, which is connected to all the worker computers 
and to whatever other gear is appropriate to the 
function of that subnetwork. Ideally, each sub- 
network has a single function such as central file 
storage, teletypewriter message handling, remote 

job entry, etc. This organization permits new 
facilities to be added to Octopus or existing 
facilities to be modified with only minimal dis- 
ruption of activities unrelated to the change. A 
highly centralized network, such as the original 
Octopus design which placed all non-worker functions 
into a single concentrator or head of the network, 
suffers because continual growth requires continual 
hardware and software modification of the head. 

This causes the head to be frequently unreliable 

or inoperative, and when the head is not functioning 
properly the entire network is unusable. The pre- 
sent trend is toward even greater decentralization: 
Eventually the network will consist of a number of 
computers, each with its own functions such as the 
worker function, file storage, message switching, 
etc. A pair of computers will be directly connected 
wherever necessary as dictated by data traffic load. 


Octopus is not designed around a particular manu- 
facture of hardware and in fact includes computers 
and other hardware from many different manufacturers. 
Only with this freedom can the network be assured 
of including the most cost-effective equipment as 
the system grows in response to changing technology. 
It is necessary for LLL to employ an engineering 
staff which designs and implements the interfaces 
between the various pieces of hardware. This has 
had the benefit that there can be close interaction 
between the hardware designers and the system 
programmers who will use the hardware, resulting 

in designs which represent excellent compromises 
between hardware complexity and software overhead, 

a situation which unfortunately does not obtain for 
many commercially available devices. 


It might be worth voicing a public complaint about 
how poorly many manufacturers understand the ways 
in which their own products are used and about the 
design shortcomings that thereby result. Many 
devices are such that it is difficult or impossible 
to insulate users and assure privacy; for example, 
multiterminal output display systems often have 
terminal selection instructions embedded in their 
display lists, thus requiring the system software 
to examine and censor all lists - a considerable 
overhead. Other devices lack obviously important 
status registers; for example, there is a paged 
processor that does not provide the offending 
virtual address at the time of a page fault, thus 
requiring interpretive reexecution of the trapped 
instruction (which, moreover, is not always pos- 
sible). Many large storage devices have unaccept- 
ably small random access times, since they combine 
very slow mechanical fetching mechanisms with 
negligible facilities for overlapping operations(5]. 


A related problem is that of useful hardware that 
is not manufactured at all. For example, because 
of the large amounts of data involved, Octopus 
must use rotating storage for buffering information 
while it awaits its turn to move further through 
the network over a particular channel. Ineffi- 
ciencies arise because of delays while read/write 
heads are moved back and forth between the area 
being written and the area being read. A disk that 
could be simultaneously read and written via two 
independently moveable heads would seem to con- 
stitute a reasonable and economic solution to the 
problem. 


6. PROGRAMMING TECHNIQUES 
Defensive programming seems to be the most important 


programming technique in Octopus. No computer in 
the network should rely on data received from 


another computer to the extent that errors in that 
data could affect activities unrelated to the data 
or could cause the computer to malfunction. The 
same rule should apply to some extent between 
programming modules in a single computer, particu- 
larly when the modules are written by different 
persons. A computer being out of service should 
inhibit only those functions for which that computer 
is essential. 


There are other programming concepts which seem to 
receive an inordinate amount of attention in the 
programming literature and should therefore be 
mentioned chiefly because they are only partially 
applied, or are not applied at all, in Octopus. 
Octopus system programs are written either in 
assembly language (more frequently without macros 
than with them) or in an LLL-designed, Fortran- 
derived language called LLLTRANLG], The system 
programmers tend to divide somewhat sharply into 
advocates of the two classes of languages, and a 
consensus does not exist. It seems fair to say, 
however, that there is no objective evidence that 
the programs written in the higher-level and lower- 
level languages differ significantly in logical 
quality, understandability, speed of programming, 
or adaptability to new situations; it is mainly a 
question of what is familiar to a given programmer. 


It might seem reasonable to suppose that, if the 
entire system were written in a single language, 
there would be less difficulty in transferring a 
program from one programmer to another. This view 
of course is based on the dubious assumption that 
programmers can be skilled in only one language. 
Moreover, at LLL, there is relatively little trans- 
ferring of programs because there is relatively 
system programmer turnover. This situation, which 
apparently does not obtain at most installations, 
is probably not unrelated to the LLL policy of 
offering considerable programmer responsibility and 
freedom (such as in the choosing of the language to 
be used). There is no two-level structure of system 
analysts and programmers. A single person, working 
closely with the small group involved in the same 
project, designs and implements a given program or 
routine. All that is required is adherence to a 
few simple standard Octopus conventions and 
protocols. 


There is no widespread feeling among Octopus pro- 
grammers that the "go to" construction is a source 
of difficulty or that every program must be block- 
structured. The problem of deadlocks (in which 

two interrelated activities await interminably for 
one another to proceed) has rarely manifested itself 
and, when it has, has been quickly discovered and 
resolved. Little, if any, interest has been aroused 
by the possibility of "proving" that the system is 
“correct.'' Such an effort would appear to be pro- 
hibitively time-consuming and would involve proofs 
whose own correctness would be suspect. The best 
guarantees of good software seem to be careful 
original design, conscientious observation during 
use, and prompt debugging. Experience has shown, 
moreover, that hardware failure is a greater problem 
than software failure. Both are subject to design 
flaws, but software once fixed remains fixed, while 
hardware continues to fatigue and wear out. 


Since the Octopus programmers are present at the 
site where their system is used and can therefore 
observe the running system firsthand, little simu- 
lation is done. It is much easier and more certain 
to simply try out a new idea, preferably in a 
limited way (e.g., on one worker), It is important 
that a thorough effort be made to foresee and pro- 
vide for all possible anomalies that may occur when 
a new program is executed but it has been found 
unprofitable to include in the initial design de- 
tailed automated responses to all unusual conditions, 
since it is very difficult to predict which com- 
binations of unusual conditions will arise in 
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practice. For example, if a processor proves so 
reliable that it seldom if ever experiences a hard- 
ware failure requiring a complete deadstart, it is 
not worth the effort required to make the deadstart 
soft (i.e., such that users experience minimal dis- 
comfort). On the other hand, if hardware failure 
proves to be a serious matter, it then becomes 
important to provide software defenses against its 
consequences. We have found that nearly all hard- 
ware failures result in highly erratic behavior that 
is readily apparent to users and operators. Quick 
observation of failures is enhanced by software- 
generated messages when anomalies are noted. We 
have not found, however, that it is worthwhile to 
design elaborate defenses against particular con- 
sequences of failure; we do not, for example, make 
all decisions relating to critical matters in two 
independent places. 


Our own observation as to the area which must re- 
receive the most attention when designing a network 
is the matter of net effective data transfer rates. 
It appears that all components eventually become 
overloaded and must be augmented or replaced. The 
issue which should most dominate a designer's think- 
ing is arranging efficient use of available capacity 
so as to prolong the life of the parts of the 
system. 


7. STANDARDS 


Octopus network standards have been selected by a 
learning process. The first time a type of facility 
is put into the network, protocols and formats are 
designed ad hoc by the engineers and programmers. 
Then, whenever it becomes clear that several faci- 
lities are being developed that have common charac- 
teristics, a standard is adopted to satisfy them 
all (as well as any foreseeable future developments), 
even if this means reworking some software already 
implemented. Any redundant effort implied by this 
approach is more than compensated for by the fact 
that design is not hampered nor degraded by adher- 
ence to standards adopted before the real nature of 
the problem had been observed. This principle, as 
well as several others noted previously, may be 
summarized by stating that Octopus uses an experi- 
mental approach. We do not adopt nor even define 
general principles before we have actually had 
experience with the situations to which they apply. 
This approach is of course made easier because of 
reliance on locally designed software and, to some 
extent, hardware, which are readily modifiable. 


There are Octopus standards for interface protocols, 
message headings, file formats, command languages, 
and many other matters. One standard may be of 
particular interest: All network messages consist- 
ing of characters are in ASCII embedded in 8-bit 
bytes, even though no large computer prior to the 
STAR-100 is oriented toward 8-bit units. This is 
an example of setting a standard based on an esti- 
mate as to what most computers will be like in 
years to come. In many Octopus computers, the 
ASCII is repacked (with suitable escape conventions) 
into 7- or 6-bit fields, whatever is most conven- 
ient to the word-size and instruction set. (The 
ASCII is never converted to EBCDIC or other code, 
which would seem to be a complete waste of effort.) 


8. RESOURCES 

Probably the most important observation about 
Octopus is that it works; it improves the speed, 
efficiency, and quality of computation at LLL and 
thereby of most activity at LLL. This does not 
mean that all users of the system are fully satis- 
fied, the reason being that, for Octopus as for 

all systems, resources are limited. The large 
hardware inventory summarized at the beginning of 
this article does not imply that Octopus users have 
access to practically unlimited computing power and 
storage capacity. The hard fact is that even - no, 


especially - at large installations it is econo- 
mically unsound to supply computing capacity much in 
excess of actual need. Those who suggest that 
modern computers have such a surfeit of resources 
that computational efficiency can be indiscriminat- 
ely sacrificed to achieve allegedly higher purposes 
are not fully in touch with all practical realities. 
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